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fln  RpproKimate  Solution  Technique  for  the  Constrained  Search  Path 

Mouing  Target  Search  Problem 

James  N.  Eagle 

James  R.  Vee 

Department  of  Operations  Research 

Naval  Postgraduate  School 

llonterey,  CR  93943 

R  discrete  time  search  is  conducted  for  a  target  mouing  among  a 
finite  set  of  eel  Is  C  =  {1, . . . ,N}.  Rt  the  beginning  of  each  time 
period  one  cell  is  searched,   If  cell  i  mas  searched  in  the  previous 
time  period,  the  current  search  cell  must  be  selected  from  the  set 
Cj  c.  C.   If  the  target  is  in  the  selected  cell  k,  it  is  detected 
(i.e.,  found)  with  probability  q|<€[QJ].   If  the  target  is  not  in 
the  cell  searched,  it  can  not  be  detected  during  the  current  time 
period.  After  an  unsuccessful  search,  a  target  in  cell  i  moves  to 
cell  j  with  probability  y^    for  the  next  time  period.  The  transition 
probability  matrix  T=  [^j;]  and  the  initial  distribution  of  the 
target  over  the  search  cells  are  known  to  the  searcher.  The 
objective  of  the  searcher  is  to  select  a  T-time  period  search  path 
which  minimizes  the  probability  of  nondetect ion. 

1.  Background 

The  path  constrained  search  problem,  described  above,  is  a 
difficult  one  to  solve  efficiently.  Trummel  and  Uei singer  [1985] 
showed  that  the  path  constrained  search  problem  with  a  stationary 
target  is  NP-complete.   The  moving  target  problem,  which  is  a 


generalization  of  the  stationary  target  problem,  is  then  also  HP- 
complete. 

Other  than  total  enumeration  of  all  search  paths,  the  only 
optimal  solution  technique  mentioned  in  the  literature  for  the 
moving  target  constrained  search  problem  has  been  the  dynamic 
programming  procedure  of  Eagle[1964a] .  Although  this  method  can 
solve  problems  much  more  quickly  than  total  enumeration,  it  can 
require  a  large  amount  of  computer  storage  as  problem  size 
increases. 

It  was  the  difficulty  experienced  with  solving  large  problems 
optimally  that  motivated  the  development  of  good  suboptimal  solution 
procedures.  The  first  such  method  proposed  mas  a  modified  branch- 
and-bound  method  by  Stewart  [1979].  Stewart  used  a  discrete  version 
of  a  moving  target  search  algorithm  given  by  Brown  [1960]  to  provide 
bounds  for  his  procedure.  However,  Brown's  algorithm  does  not 
necessarily  give  optimal  solutions  when  search  effort  is  discrete, 
so  these  "bounds"  may  result  in  an  optimal  branch  of  the  enumeration 
tree  being  mistakenly  fathomed.  Nonetheless,  Stewart's 
computational  experience  with  1 -dimensional  search  problems 
indicates  that  the  method  can  perform  well. 

Another  approximate  procedure  was  given  by  Eagle  [1981b].  This 
dynamic  programming  method  uses  a  moving  or  "rolling"  time  horizon 
that  greatly  reduces  the  computer  storage  requirements,   It  was  used 
to  approximately  solve  a  small  2-dimensional  problem  (3  by  3  search 
grid)  for  10  time  periods.   This  procedure  generalizes  myopic  search 
by  selecting  in  each  time  period  the  next  cell  to  be  searched  under 
the  assumption  that  the  search  ends  m  time  periods  in  the  future. 


For  myopic  search,  m  is  1.  For  small  enough  m,  this  procedure  can 
be  implemented  on  a  microcomputer. 

Reported  here  is  a  third  sub-optimal  solution  method  which,  like 
Brown's  algorithm,  is  derived  from  a  nonlinear  programming 
formulation  of  the  search  problem.  Unlike  the  problem  addressed  by 
Brown,  however,  this  formulation  (a)  allows  for  path  constraints  to 
be  specified  for  the  searcher,  (b)  does  not  allow  search  effort  in 
each  time  period  to  be  infinitely  divisible  over  the  search  cells, 
and  (c)  does  not  have  a  convex  detection  function.   Consequently 
the  objective  function  of  the  nonlinear  program  is  not  necessarily 
convex  and  the  solutions  obtained  may  be  local  rather  than  global 
optima.  But  like  Brown's  method,  the  structure  of  the  problem 
allows  a  simple  implementation  of  the  nonlinear  programming  solution 
technique.  Uhen  considered  without  path  constraints,  this  procedure 
has  similarities  to  discrete  versions  of  both  Brown's  algorithm  and 
those  of  Washburn  [1980]  and  [1983], 

2.  Definitions 

The  movement  of  the  searcher  is  described  by  a  nonhomogeneous 
flarkov  process.  Let  Sjj(t)  be  the  probability  that  the  searcher  will 

search  cell  j  in  the  time  period  t,  given  that  cell  i  was  searched  in 
time  period  t-1 .  Then  a  search  plan.  S  =  {S(1),...,S(T)},  is  a 
sequence  of  T  NxN  stochastic  matrices  satisfying 

Zj€Cj  Sy(t)  =  1,    i  =  1,...,N;  t  =  1,...,T. 


fl  deterministic  search  plan  is  a  S  composed  entirely  of  ones 
and  zeros. 

Pjj(t)  is  the  joint  probability  that,  after  the  search  and 

target  transition  in  time  period  t,  the  searcher  is  in  cell  i  and 

the  target  is  in  cell  j  and  has  not  been  detected  by  the  first  t 
searches. 

p(0)  €  !RNxN  is  the  initial  joint  searcher-target  distribution 
and  is  assumed  to  be  known  by  the  searcher. 

Hate  that  Ejj  pjj(t)  is  the  probability  that  the  target  has  not  been 

detected  by  the  searches  conducted  in  time  periods  1  through  t, 
Also,  Pjj(t)  can  be  calculated  recursively  from  pjj(t-1)by 

conditioning  on  the  searcher  cell  and  the  target  cell,  after  the 
search  and  target  transition  in  time  period  (t-1).  Specifically, 

Pij(t)     =     (l-q^jj)  £w  PW(t-DSkj(t)ru  ,  (0 

where  d\\   is   1    if    i  =  j    and  0    if    i*j. 

3.  The  Search  Problem  as  a  Nonlinear  Program  (NLP) 

We  seek  the  solution  of  the  following  NLP 

subject   to 

Sjj(t)    =0,  i   =   \,...,H}   j^Cjj   t   =   I, ...J  (1) 

S(t)    >,  Q,         1   =   1 ,  ..... T  (5) 


where  Py(T)  is  calculated  recursively  from  (1).  The  decision 
variables  are  the  NxN  matrices  SO ), . . . ,S(T) .  To  solve  this 
problem,  p(0),  (qj,  ...^h)*  and  r  must  be  specified, 

The  four  propositions  which  follow  establish  certain  properties 
of  the  above  nonlinear  program.   It  is  noted  that  this  problem, 
while  having  a  relatively  complicated  objective  function,  has 
constraints  which  are  linear  and  highly  structured. 

Proposition  1:  The  minimum  of  nonlinear  program  (2)-(5)  is  achieved 
by  a  deterministic  search  plan. 

Proof:  Let  S  be  an  optimal  search  plan.   (Such  a  plan  exists 

since  the  objective  function  (2)  is  continuous  in  S  and  the 
set  of  feasible  S  defined  by  (3)-(5)  is  compact.)  S  defines  a 

T-time  period,  N-cell  Narkov  process  defining  probabi I ist icly 
an  optimal  search  path.  Let  {£fj...j£z}  De  tne  finite  se*  of 
possible  deterministic  search  paths  generated  by  S  .  Each  (.\, 
has  an  associated  ak  €  IRNx1  where  the  j   component  of  a^  is  the 
probability  of  nondetect ion  given  the  searcher  follows  £|<  and 

the  target  starts  in  cell  j.  Also  let 
Tlj  =  £,  Psj(O) 

be  the  probability  that  the  target  starts  in  cell  j.  Then  the 
probability  of  nondetect ion  given  fj,  is  followed  and  an  initial 
target  distribution  of  T|€lR1xM  is  the  dot  product  T|ak.  How  if 
P(£k)  is  the  probability  of  the  searcher  following  path  fk  when 
S  is  used,  we  have 
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Prob{nondetection|  S*,T)  }  =  V  P(fk)  T|ak 

1    r\a*t 
where 

a*   =  argmina     (T]ak). 


So  the  deterministic  search  path  associated  with  a*  (and  the 
deterministic  search  plan  which  generated  it)  is  also  optimal. 

Proposition  2:  S  is  a  deterministic  search  plan  if  and  only  if  S  is 
an  extreme  point  of  the  linear  constraints  (3)- (5). 

Proof:   If  S  is  deterministic,  then  by  definition  all  component 
matrices  of  S  must  be  composed  entirely  of  zeros  and  ones. 
Such  an  S  can  not  be  written  as  a  3trict  convex  combination  of 
two  other  matrix  series  satisfying  (3)-(5).  So  S  is  an 
extreme  point.  Find  if  S  is  not  deterministic,  then  some 
component  matrix  must  contain  a  row  uector  with  two  or  more 
components  strictly  between  0  and  1,  This  row  vector  can  then 
be  expressed  as  a  strict  convex  combination  of  two  other 
distinct  and  feasible  row  vectors.  Thus  S  is  not  an  extreme 
point . 

Proposition  3:  The  objective  function  (2)  is  linear  in  S  when 
constrained  to  any  edge  of  the  simplex  formed  by  (3)- (5). 

Proof:  Lets'  and  S"  be  any  two  adjacent  extreme  point  solutions 
of  the  constraints  (3) -(5).   Specifically,  S"  and  S"  are 


identical  stochastic  (OJ)  matrix  series  except  in  one  row  of 
one  matrix.  Let  t  be  the  time  period  where  S'  and  S"  differ, 
and  for  any  A€[0, 1 ]  let 

Sk   =  AS'  *   (1-A)S" 

=  S(1 ),..., S(t-1),AS'(t)  +  (1-A)S"(t),S(t  +  1 ),..., S(T) 
Conditioning  on  the  searcher's  cell  and  the  target's  cell 
before  the  search  in  time  period  t,  we  can  write  the  objective 
function  (2)  evaluated  at  Sk   as 

Zu  EmPu^-^  [(1-qkd|i)$Vt)7ji]  Qkl(t  +  1),   (6) 
where  <5y  is  as  in  (1),  and  Qy(t+1)  is  the  probability  that  the 

target  is  undetected  by  searches  in  time  periods  t+1,...,T 
given  the  searcher  is  in  cell  k  and  target  is  in  cell  I  at  the 

end  of  time  period  t.  Uniting 

S\(t)  =  AS'ik(t)  ♦  (1-A)S"ik(t), 

and  observing  that  pjj(t-l)  and  Qki(t  +  1)  ore   not  functions  of  A, 
shows  (6)  to  be  linear  in  A. 

Proposition  4:  Rll  basic  feasible,  solutions  to  the  non I  i near- 
program  (2) -(5)  are  nondegenenate. 

Pnoof:  The  constraint  matrix  represented  by  (3)  and  H)  consists 
o f  NT  I i  near  I y  i  ndependent  rows .  So  bas  i  c  so  I ut  i  ons  will  a  I  ways 
have  NT  basic  variables.  Ue  argue  that  these  basic  variables 
will  always  be  positive.  Each  constraint  consists  of  nonnegat ive 
variables  summing  to  unity.  So  any  feasible  solution  must 
i  nvo  I  ve  at  I  east  one  var  i  ab  I  e  from  each  const  v-q  int.   Si  nee  t  here 
are   NT  const  ra  i  nt  s  and  NT  bas  i  c  oar i  ab I es ,  any  bas  i  c  so  I ut  i  on 


must  involve  a  single  basic  variable  in  each  constraint.  Rnd 
furthermore,  since  the  constraints  sum  to  unity,  the  value  of 
each  basic  variable  must  be  1, 

4.  Applying  the  ConueH  Simplex  Method  (CSM) 

The  CSM,  which  is  a  generalization  of  the  simplex  method,  is 
applied  to.  nonlinear  programs  mith  linear  constraints.   If  the 
current  feasible  solution  is  an  extreme  point  of  the  constraints, 
then  the  CSI1  determines  the  rate  of  change  of  the  objective  function 
along  the  edges  radiating  from  that  extreme  point.  The  current 
solution  then  moves  along  an  edge  mith  the  greatest  initial  rate  of 
improvement.  This  movement  continues  until  a  local  optimum  is  found 
along  that  edge  or  an  adjacent  extreme  point  is  reached  (whichever 
occurs  first).   If  there  is  no  edge  along  which  the  objective 
function  can  be  improved,  then  the  current  extreme  point  is  a  Kuhn- 
Tucker  point,  and  a  local  optimum  has  been  found.  The  importance  of 
Propoposit ion  1  is  that  it  guarantees  that  an  extreme  point  which 
does  not  satisfy  the  Kuhn-Tucker  conditions  can  always  be  improved 
with  a  single  iteration  of  the  CSI1. 

The  CSH  is  applicable  to  the  nonlinear  program  (2)-(5)  because 
the  constraints  are   linear.  Furthermore,  its  implementation  is 
especially  easy  since  from  Proposition  3  the  objective  function  is 
linear  along  edges  of  the  simplex.   Thus  once  movement  along  an  edge 
is  determined  to  reduce  the  objective  function,  this  movement 
continues  until  an  adjacent  extreme  point  is  reached.   That  is,  no 
one-dimensional  searches  for  local  minima  along  extreme  edges  are 
required. 


The  CSfl  can  thus  be  implemented  for  this  problem  by  starting  at 
any  feasible  extreme  point,  evaluating  the  objective  function  at  all 
adjacent  extreme  points,  and  moving  directly  to  the  adjacent  extreme 
point  with  the  smallest  objective  function  value. 

The  calculations  required  to  evaluate  the  objective  function 
(2)  at  any  extreme  point  of  the  constraints  are  simplified  when  the 
searcher's  starting  cell  is  known  with  certainty  (the  usual  case  for 
most  search  applications).  This  occurs  because  the  searcher's 
starting  cell,  together  with  the  deterministic  search  plan 
associated  with  an  extreme  point,  uniquely  determine  a  search  oath 
s  =  (s(1 ), . . . ,s(T)),  where  s(t)  is  the  searcher's  cell  in  time 
period  t.  find  the  probability  of  nondetect ion  given  s  is 

T 

PND(s)  =  Tl(nrs(t))1  (?) 

t=i 

where  r\   is  the  initial  target  distribution,  1  is  a  column  vector  of 
ones,  and  T£(t)  is  the  target  transition  matrix  T  with  row  s(t) 
mult  ipl  ied  by  (1-qs(t)) . 

It  is  also,  observed  that,  although  a  general  search  plan  S  is 
an  element  of  IRTxNxN,  a  deterministic  search  plan  can  be  represented 

by  a  TxH  integer  matrix  S.  Each  element  5(t,j)  is  the  searcher' 5 
cell  in  time  per  i  od  t  g  i  ven  t  he  searcher ' s  cell  i  n  t  i  me  per  i  od  t - 1 
was  j.   S  represents  a  feasible  search  plan  if  and  only  if  each 
5(t,j)  is  an  element  of  Cj. 


The  CSN,  specifically  tailored  for  the  search  problem  in  (2)- 
(5),  is  the  fol lowing: 

1.  Specify  s(1)  and  an  initial  feasible  5. 

2.  Determine  s  =  (s(1 ), . . . ,s(T))  from  s(1)  and  S. 

3.  Calculate  PHD(s)  from  (7)  and  set  PHDmin  =  PHD(s), 

4.  For  t=1  to  T, 

a.  Uary  5(t,s(t))  over  all  Cs(tj  to  generate 

candidate  adjacent  extreme  points  S'. 

b.  For  each  5',  calculate  the  new  search  path  s' 

and  PND(s'). 

c.  If  PND(s')  <  PHDmJn,  save  S"'  and  s\  and  set 
PNDmin=PND(s'). 

d.  Cont  inue  to  next  t . 

5.  If  no  improvement  is  achieved  in  PHDrrijn  after  t=T,  then 

STOP.  Otherwise  define  5  as  S',  s  as  s',  and  return 
to  step  4. 

Note  that  each  main  iteration  of  this  algorithm  (i.e., step  1.) 
requires  the  calculation  of  PND  for  2j  |Cj|  search  paths,  where  ICJ 
is  the  number  of  cells  in  Cj. 
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5.  The  Starting  Solution 

In  this  section,  a  procedure  for  generating  an  initial  feasible 
S  is  described,  fl  reasonable  S  would  be  one  that  corresponds  to  a 
myopic  search  path,  fl  myopic  search  path  is  a  path  where  the 
searcher  moves  from  the  cell  searched  in  time  period  t  to  that 
accessible  cell  j  with  the  largest  qjT)j(t),  where 

rjj(t)  =  2j  p^t) 

=  Prob{target  in  cell  j  and  has  not  been 
detected  by  searches  in  time  periods  1,.,.,t}. 

If  two  or  more  accessible  cells  have  identical  maximal  values 
of  qiT)j(t),  then  the  myopic  policy,  as  used  here,  attempts  to  break 

the  tie  by  selecting  a  cell  with  the  minimum  Euclidean  distance  (or 
some  other  reasonable  norm)  to  a  cell  with  the  maximum  value  of 
qjT"|j(t).   If  a  tie  still  exists,  it  is  broken  randomly.  Uhen  the 

target  is  distant,  this  tie  breaking  procedure  attempts  to  generally 
move  the  searcher  towards  a  favorable  cell  and,  hopefully,  into 
position  for  a  future  detection. 

fl  procedure  is  still  needed  to  complete  the  initial  feasible  S.  One 
procedure  would  be  to  assume  that  the  target  distribution  is  a  stationary 
distribution  generated  by  the  target  transition  matrix  T.  (That  is,  the 
target  distribution  isTjelR   where  r\   =  T)r,  T\\  %  0 . )   Then  for  each  time 
period  (i.e.,  each  row  of  S)  the  myopic  policy  is  used  to  determine  the 
next  cell  to  be  searched.  This  results  in  an  S  which,  except  for  one 
element  in  each  row,  has  identical  rows.   The  element  which  is  different 
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corresponds  to  the  initial  myopic  search  path.  This  starting  solution  mas 
tested  and,  for  the  problems  examined,  appeared  to  work  well. 

6.  Three  Examples 
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Figure  I.  25  cell  search  grid. 

The  target  and  searcher  move  among  the  25  cells  of  Figure  1. 
In  the  next  time  period,  the  searcher  has  access  to  the  cell  just 
searched  plus  all  adjacent  cells.  Cells  are  adjacent  if  they  share 
a  common  side.  Thus,  for  example,  0j  =  {1,2,6}  and 
C3  =  {2,3,4,6}. 

In  each  target  transition,  the  target  remains  in  the  previously 
occupied  cell  j  with  probability  .4  and  mooes  to  an  adjacent  cell 
with  probability  .6/irij,  where  uij  is  the  number  of  cells  adjacent  to 

ce  I  I  j  . 

In  the  first  two  examples,  the  searcher  starts  in  cell  1  and 
the  target  in  cell  13.  Assume  that  detection  is  certain  if  the 
target's  cell  is  searched.  That  is,  qk  =  1,  k  =  1,2,..., 25. 
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fl  10-time  period  problem  mas  solved  using  FORTRRN  77  on  an  IBfl 
3033  mainframe  computer.  The  following  results  were  obtained: 

Prob.  of        CPU  time 
Search  path  Nondetection       (sec) 

Myopic   6  ?  12  13  14  13  18  19  14  9     .497?  .1 

CSM    6  7  8  13  14  19  18  17  12  7     .4886  1.42 

In  addition,  a  total  enumeration  routine  was  written  to  find 
the  optimal  solutions.  This  procedure  required  25  CPU  minutes  to 
examine  the  1,225,623  possible  10-time  period  search  paths,  and 
showed  the  CSI1  solution  to  be  one  of  12  distinct  optimal  solutions. 

In  this  example  the  myopic  policy  was  close  to  optimal.  Using 
the  CSfl  resulted  in  only  a  slight  improvement,  while  requiring 
considerably  more  CPU  time.  The  next  examples  show  these 
observations  are  not  true  in  general. 

In  the  second  example,  a  "fast"  target  was  considered. 
Specifically,  the  probability  that  the  target  in  cell  j  remained  in 
j  was  0,  and  the  probability  that  a  transition  occured  to  any  of  the 
mf  adjacent  cells  was  1/mj.  Rll  other  problem  parameters  remained 

the  same.  This  problem  gaive  the  following  results: 

Prob.  of         CPU  time 
Search  path  Nondetection        (sec) 

flyopic       6  7   12   12     7     8     9   14   19  18  .4540  .1 

CSM  67     7     6     91419181712  .3868  1  .73 

Rgain,  total  enumeration  showed  the  CSI1  solution  to  be  optimal. 
The  third  example  examined  the  possible  consequences  of  a 
moving  mean  target  position.  Here  the  target  started  in  cell  21  and 
moved  up  or  right  one  cell,  each  with  probability  ,45.  The  target 


\\ 


Prob.  of 

CPU  time 

Nondetect  ion 

(sec) 

.2517 

.1 

.1531 

1.09 

remained  in  its  current  cell  with  probability  , 1.  This  movement  up 
and  right  continued  until  either  the  top  or  right  boundary  was 
reached.  Then  with  probability  .9,  the  target  moved  one  cell  up  (if 
currently  in  cell  25,  20  15,  or  10)  or  one  cell  to  the  right  (if  in 
cell  1,  2,  3,  or  1) .  Rgain,  the  target  remained  in  the  current  cell 
with  probability  .1.  Uhen  the  target  reached  cell  5,  it  remained 
there  forever  and  was  assumed  to  have  escaped.  Except  in  cell  5, 
detection  was  certain  if  the  target's  cell  was  searched.   In  cell  5 
the  target  could  not  be  detected.  For  the  10-time  period  problem, 
the  following  results  were  obtained: 

Search  path 
Myopic   6  11  11  12  13  13  11  15  15  10 
CSM    6  11  11  12  13  11  15  15  10  10 

Total  enumeration  again  showed  the  CSI1  path  to  be  optimal,  In  this 
final  example,  the  stationary  target  distribution  has  the  target  in 
cell  5  (the  trapping  state)  with  certainty.   Since  q5  is  0,  the 

modified  myopic  procedure  given  in  the  previous  section  fails  to 
give  a  unique  starting  S.   To  find  a  starting  solution  for  this 
example,  it  was  assumed  that  q5  was  an  arbitrarily  small,  positive 
number. 

Finally  it  is  noted  that,  like  most  nonlinear  programming 
solution  procedures,  using  a  poor  starting  solution  can  result  in  a 
poor  final  solution.   In  the  last  example,  setting  S-g  =  j  (except 

for  the  one  element  of  each  row  determined  by  the  myopic  path) 
resulted  in  a  local  optimal  PHD  of  .2379,  a  slight  improvement  over 
the  myopic  solut  ion. 
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